Self-Supervised Learning for Semi-Supervised Temporal Language Grounding

Given a text description, Temporal Language Grounding (TLG) aims to localize temporal boundaries of the segments that contain specified semantics in an untrimmed video. TLG is inherently challenging task, as it requires comprehensive understanding both sentence and video contents. Previous works either tackle this task fully-supervised setting large amount annotations or weakly-supervised usually cannot achieve satisfactory performance. Since manual are expensive, cope with limited annotations, we semi-supervised way by incorporating self-supervised learning, propose S elf- upervised emi- xmlns:xlink="http://www.w3.org/1999/xlink">T emporal xmlns:xlink="http://www.w3.org/1999/xlink">L anguage xmlns:xlink="http://www.w3.org/1999/xlink">G rounding (S $^{4}$ TLG). S consists two parts: (1) A pseudo label generation module adaptively produces instant labels for unlabeled samples based on predictions from teacher model; (2) feature learning inter-modal intra-modal contrastive losses learn representations under constraints content consistency video-text alignment. We conduct extensive experiments ActivityNet-CD-OOD Charades-CD-OOD datasets. The results demonstrate our proposed can competitive performance compared state-of-the-art methods while only requiring small portion annotations.